Fuzzy protection of visual content

ABSTRACT

A method and system for scrambling visual information are, described. An image, preferably in an incompressed format, is divided into a number of patterns having a different aspect ratio. Each pattern is mapped on a matrix i.e positioned on X and Y-axis. The patterns are then arranged in a different and new shape for e.g. a rectangle, preferably, fitting the size of manoblocks in a MPEG video, to that of the original shape of the image but having the same number of patterns or same area. The patterns of pixels are then relocated in the new shape and a matrix is again mapped for the new arrangement of patterns. A key is then generated comprising the information of the encrypted and earlier image. The player decrypts the encrypted visual content or image by means of the information provided by the key.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/566,458, filed Aug. 3, 2012; which is a divisional of U.S. patentapplication Ser. No. 12/527,059, filed Aug. 13, 2009, which is a 35U.S.C. 371 national stage filing from International Application No.PCT/IB2008/000334 filed Feb. 14, 2008 which claims priority to IndianApplication No. 307/DEL/2007 filed Feb. 14, 2007, the entire contents ofeach of which are incorporated herein by reference.

DESCRIPTION FIELD OF THE INVENTION

The instant invention relates to the field of image processing,encryption and decryption of digital visual content and in particular tofuzzy protection of visual content by disallowing unauthorized access ofvisual content. The present invention is also related to Coder/Decoderapplication in the field of image processing.

BACKGROUND OF THE INVENTION

Entertainment industry includes movie, cable TV, screenplays etc. Moviehas been a major industry in the world of entertainment. The productionof a movie requires a huge sum of money, talent of people and a greatamount of effort. These efforts, money and talent of too many peopleinvolved in the formation of a movie or a screenplay bear no fruits atthe end if copied or accessed by unauthorized people. Also, there areconfidential images or moving pictures, which might be of ultimatesecrecy to the welfare of the nation and requires strong security. Thus,in order to protect such crucial visual information from unauthorizedaccesses, a method and a system for securing digital visual content arein core need.

In present times various techniques have been used to stop copying ofvisual information. For instance, image watermarking is the process bywhich some information is inserted within a host image, e.g. to enablecopyright protection or image authentication. Several, oftenconflicting, requirements are imposed on the insertion mechanism, suchas invisibility, robustness, high information content, and fast andreliable detection.

However, so far there have been no conventional techniques or system todisallow viewing of secured visual information to an unauthorizedperson.

Hence, there is a strong need for a system and method which disallowsviewing of important and sensitive visual information by an unauthorizedperson.

Television encryption, often referred to as “scrambling”, is used tocontrol access to pay television services, usually cable or satellitetelevision services. Pay television exists to make revenue fromsubscribers and sometimes those subscribers don't pay. The prevention ofpiracy on cable and satellite networks has been one of the main factorsin the development of Pay TV encryption systems.

The early cable based Pay-TV networks used no security. This led toproblems with people connecting to the network without bothering to pay.Consequently, some methods were developed to frustrate theseself-connectors. The early Pay-TV systems for cable television werebased on a number of simple measures. The most common of these was achannel based filter that would effectively stop the channel beingreceived by those who had not subscribed. These filters would be addedor removed according to the subscription. As the number of televisionchannels on these cable networks grew, the filter based approach becameincreasingly impractical.

Other techniques such as adding an interfering signal to the video oraudio began to be used as the simple filter solutions were easilybypassed. As the technology evolved, addressable set top boxes becamecommon and more complex scrambling techniques such as digital encryptionof the audio or video cut and rotate (where a line of video is cut at aparticular point and the two parts are then reordered around this point)were applied to signals.

Encryption was used to protect satellite distributed feeds for cabletelevision networks. Some of the systems used for cable feeddistribution were expensive. As the DTH market grew, less secure systemsbegan to be used. Many of these systems (such as OAK Orion) werevariants of cable television scrambling systems that affected thesynchronization part of the video, inverted the video signal or added aninterfering frequency to the video. All of these analogue scramblingtechniques were easily defeated.

Usually a video player, which is a kind of media player can be used forplaying back digital video data from media such as optical discs (forexample, DVD, VCD), as well as from files of appropriate formats such asMPEG, AVI, Real Video, and QuickTime. Many of the video players alsosupport simple playback of digital audio making the content susceptibleto unauthorized access. Hence cryptographic techniques are used tosecure the same. In modern times, the study of cryptography is reliedupon for securing digital content. Cryptography is considered to be abranch of both mathematics and computer science, and is affiliatedclosely with information theory, computer security, and engineering.

Hence there is a need for a system and method to encrypt digital contentto make it secure and inaccessible to unauthorized access.

SUMMARY OF THE INVENTION

In order to achieve the above objectives and advantages, the instantinvention encrypts visual content i.e. still or moving image so as todisallow any unauthorized person to access the visual content.

Advantageously, the instant invention leads to repositioning of equalsized patterns of pixels so as to hide or encrypt the visualinformation.

The above objectives or advantages have been disclosed in thespecification so as to clarify the invention closely. However, the scopeof the above invention should not be constrained or limited by the aboveobjectives or advantages.

According to the present invention, the image is divided in to n numberof patterns each having a x b pixels. The n patterns of a x b pixels aremapped on a matrix i.e. are positioned on X and Y-axis. The n patternsare then arranged in a different and new shape for e.g. a rectangle, tothat of the original shape of the image but having the same number ofpatterns or same area. The patterns of pixels are then relocated in thenew shape and a matrix is again mapped for the new arrangement ofpatterns. A key is then generated comprising the information of theencrypted and earlier image, which is stored along with the still ormoving image file.

In order to decrypt the encrypted visual content or the encrypted image,the player reads the key or encrypted visual content and decrypts theencrypted visual content or image by means of the information providedby the key.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears. Thesame numbers are used throughout the drawings to reference like featuresand components.

FIG. 1 depicts a hardware representation of a possible integration ofthe software

FIG. 2 represents an equirectangular image

FIG. 3 is a schematic drawing representing the stream of data goingthrough the FUZZY codec

FIG. 4 a illustrates the encoding process through the fuzzy encoder

FIG. 4 b illustrates the decoding process through the fuzzy decoder

FIG. 5 a defines configuration with one group of users

FIG. 5 b defines configuration with multiple groups of users

FIGS. 6 a to 6 e represent a graphical illustration about the way theencryption key is generated

FIG. 7 represents the details of the ENCRYPTION MODULE

FIG. 8 represents the details of the REVERSE ENCRYPTION MODULE

DETAILED DESCRIPTION OF THE INVENTION

A method and system for securing digital visual information aredescribed. The system and methods are not intended to be restricted toany particular form or arrangement, or any specific embodiment, or anyspecific use, disclosed herein, since the same may be modified invarious particulars or relations without departing from the spirit orscope of the claimed invention hereinabove shown and described of whichthe apparatus or method shown is intended only for illustration anddisclosure of an operative embodiment and not to show all of the variousforms or modifications in which this invention might be embodied oroperated.

The coder part of the present invention can be used either as astandalone application or can be integrated as an existing export tool.However, the decoder part of the instant invention has to be integratedinto existing application, basically a video or multimedia player thatdeliver and display visual content to end user.

In an embodiment, an equirectengular image or movie is herein divided into equal patterns of 16×16 pixels and then are positioned in accordanceto disclosed method of the instant invention in a new texture of ratio32:1. The advantage of the instant method is brought about by therepositioning of patterns of the image and that too in a differenttexture for instance position of 16×16 pixel patterns might be changedafter every 100 patterns of 16×16 pixels.

The image in FIG. 2 represents an Equirectangular image (expressed inpixels) with 360 degrees horizontal information and 180 degrees verticalinformation. This image aspect ratio is 2:1 that mean that the width istwice larger than the height.

In FIG. 3, the SOURCE DATA in 6 is encoded through the FUZZY ENCODER in7 that delivers a multimedia file or video stream in 8. The VIDEO STREAMis then sent to the FUZZY DECODER in 9 that decodes the information, andsends it to the VIDEO RENDERER in 10.

FIG. 4 a depicts the encoding process. The Source data in 20 can be avideo file, a video stream or a still image. This data have to be amultimedia file at least understandable by an API similar to anyDirectShow Filter in 25. This source Data is described in resolutioninformation (in pixels AREA), in Compression type, in Frame Rate (inFPS) and in Color Depth. The sound information at that point is notencrypted and is not processed through the FUZZY ENCODER. The originalsound data 26 is then directly transmitted from the step 25 to the step90, where the encoded file is written. The synch information is kept in25 (which is the same API as 90) and used in 90 to write the ENCODEDDATA.

The Source Data is loaded in the Fuzzy Encoder through the FRAMEEXTRACTOR READER in 30, as the preferred process have to be computedframe by frame to prevent any data loss. This module read the incomingdata and extracts each frame separately one by one. This process is wellknown and is processed first through an API from Microsoft calledDIRECTSHOW FILTER in 25. Each frame has to be progressive to prevent anycompression anomaly.

DirectShow divides the processing of multimedia tasks such as videoplayback into a set of steps known as filters. Each filter represents astage in the processing of the data. Filters have a number of input andoutput pins which connect them together.

Then the Frame extracted in 30 is analysed in the PATTERNS READER in 40,where the whole image, expressed in pixels is converted into patterns of16×16 pixels (to fit the macroblock size fixed by existing compressioncodec used with the AVI container and MPEG formats). Then the patternslocations are stored to be scrambled into a new position by theENCRYPTION MODULE in 60.

The ENCRYPTION MODULE in 60 generates a key that provides new positionsfor each patterns extracted in 40. The key can be generated using manywell known existing possibilities such as a Keygen. The key can also begenerated manually using empiric methods. The Encryption Key is neededin 140 to re format the ENCODED DATA 100. The Encryption key isgenerated on the ENCODER device, and can be provided to the DECODERdevice using all existing methods to transport data. The Encryption keycan be embedded into the DECODER as an electronic device as well.Obviously, the key has to be delivered to the DECODER using a separateway as the ENCODED DATA to maintain the security level. Depending on thesecurity level needed, that Encryption key can also be encrypted using aknow encryption algorithm, or can be delivered as a text file.

The final ENCODED DATA 100 has exactly the same number of patterns(Macroblocks) than the SOURCE DATA 20.

Then the full frame is generated in the MACRO BLOCKS MUX (Multiplexer)in 50, using the new PATTERNS position generated using the key in 60from the sources acquired in 40. Once that step is completed, the frameis written by the FRAME WRITTER in 70 and stored into the MULTIPLE FRAMEBUFFER in 80.

The MULTIPLE FRAME BUFFER 80 function is to store the needed number offrames to compute the final compression algorithm by the DIRECTSHOWFILTER 90. The number of frames stored before compression depends on thecompression key frame required by the compression filter. It is alsoindexed with the position in the file where the encryption key changesthe position of each pattern. It is also possible that the encryptionkey is set up only at the beginning of the file, but it can also changewith a certain frequency, interlaced with existing frames.

This part completes the encryption process included in the ENCODER part.The Encoder part is usually not played on the same device as the DECODERpart. The two elements could be linked with known methods if the ENCODEDDATA in 100 is a stream data and not a recorded file.

FIG. 4 b represents the decoding process. While the ENCODED DATA istransmitted or stored on an other DEVICE to be played, the data isprocessed through the DIRECTSHOW FILTER in 110.

Then, the DIRECTSHOW FILTER is directly connected to the PATTERNS READERin 120 that format the pattern information for the MACRO BLOCKS DEMUX(Demultiplexer) in 130 where all the patterns are re-organised, usingthe encryption key imported into the REVERSE ENCRYPTION MODULE in 140.As said before, the key could be easily write in hard in the device as aROM data for example. At that point, all the patterns are re-organisedas the original SOURCE DATA (with the compression off course).

The re-organised data is sent to the DIRECTSHOW FILTER in 150 thatgenerate the DECODED DATA in 160 using existing functions.

The DATA at that point cannot be stored as a multimedia file usingdifferent filter to prevent non authorised user to have access to nonencrypted data.

The DIRECTSHOW FILTER in 110 split the AUDIO/VIDEO information, as theAUDIO doesn't have to be processed through the FUZZY DECODER part, andis directly transmitted to the output part of the DIRECTSHOW filter in150.

The DIRECTSHOW FILTER in 150 connects and synchronise together again thetwo components of the multimedia file. The video part, known here as theDECODED DATA in 160, is sent to the VIDEO RENDERER 170 that managesgraphic information for the display device. The sound part is sent tothe SOUND RENDERER in 180 that manages the sound device.

FIG. 5 a represents the case when a unique encryption key is providedand used. AU the end users of group A using the player application areallowed to use and display the content encoded by the ENCODING PLATFORM200. The only limitation is that they are allowed to display thecontent, but they are not allowed to modify it.

FIG. 5 b represents the case when multiple groups of users are present.

Each key have to be assigned to a group of content for encryption and toa group of end users for display.

Each key can be generated and encrypted on the same ENCODING PLATFORM200 or can be encoded and used on multiple ENCODING Platforms.

Let's say that we have N groups of different users. Each one has anencryption key different. The Key A is dedicated to the group A, the keyB is dedicated to the group B, and the key N is dedicated to the groupN. The restriction between different keys and then different groups arenot restricted.

The common case should be that Group A have only access to the mediaencrypted with the key A, and the Group B has only access to the mediaencrypted with the key B, and so on.

FIGS. 6 a to 6 e are provided as an example only. In this example, weconsider that the SOURCE DATA is already extracted and is a progressivescan equirectangular frame, with a size of 1024 pixels width and 512pixels height. The process is integrated as a loop, and each frame haveto be processed one by one before compression.

FIG. 6 a shows an equirectangular image of 1024×512 Pixels i.e. 2048patterns of 16×16 pixels. This frame represents the typical image thatcould be extracted by the FRAME EXTRACTOR READER 30.

FIG. 6 b represents a graphic representation of the PATTERN extractionfunction working in the PATTERNS READER 40. Therefore, the length of theimage comprises of 64 patterns of 16×16 pixels and width comprises of 32patterns of 16×16 pixels.

FIG. 6 c is a matrix formed so as to represent the positioning of eachpattern on X and Y-axis. Thus, by means of a matrix each pattern of16×16 pixels is given a coordinate as (X, Y).

FIG. 6 d represents the new matrix after encryption process 320, shownin FIG. 6 e and FIG. 7.

In FIG. 6 e, a new shape with the same number of patterns accommodationfacility is created i.e. in present embodiment the new shape comprisesof 2048 pattern accommodation capacity, which is equivalent to that ofthe original, unencrypted image shape's pattern accommodation capacity.The new shape in which the patterns are relocated is a rectangularshape. However, the area covered by the patterns remains the same.Herein the embodiment the new shape is a rectangle having 256 patternsalong the length and 8 patterns along the width. Thus, herein the numberof patterns along the length and number of patterns along the width isin the ratio of 32:1. In step 4 the patterns are relocated in the newshape that is the rectangular shape, the patterns are relocated in aspecial order by applying a fuzzy algorithm.

FIG. 7 represents the detail of the ENCRYPTION MODULE 60 of FIG. 4 a,where the ORIGINAL MATRIX POSITION 300 receives the informationregarding the patterns through the PATTERNS READER 40. The informationcollected is the number of patterns and their original location into thematrix. This information is described as the MATRIX. These matrixvariables are then scrambled in the PATTERN POSITION CHANGER 310 arerecombined into a new matrix in the NEW MATRIX POSITION 320. The way thevariables positions are scrambled can be managed manually by the enduser that manages the ENCODING process. It can be as well imported as atext file following the right formatting. Then, the new’ matrix positioninformation is sent to the KEY GENERATION 330 and to the MACRO BLOCKSMUX 50. The KEY GENERATION 330 generate a encryption key, based on thenew matrix position information, using well known existing methods. Thatkey source is then saved as a METADATA ENCRYPTION KEY 65, to be usedlater on the Player device.

FIG. 8 represents the detail of the REVERSE ENCRYPTION MODULE 140 ofFIG. 4 b. The KEY DECODER 400 receives the METADATA ENCRYPTION KEY 65,where the process re-creates the original matrix information relative tothe patterns positions in the frame. The PATTERN POSITION CHANGER 410receives the incoming pattern position from the PATTERNS READER 120 withthe KEY DECODER information. This provides the original matrix positionin ORIGINAL MATRIX POSITION in 420. This information is then sent to theMACRO BLOCKS DEMUX in 130 in order to generate the original frame to bemanaged by the DIRECTSHOW FILTER in 150. The key information is notrestricted to the pattern position in the frame.

EMBODIMENT OF THE INVENTION

The system CODEC (Coder Decoder) is then combining two separated parts:

-   -   The Coder that encode the raw video file or stream using an        encryption key,    -   The Decoder that decode the encrypted data using the original        encryption key information, and deliver the video file        reconverted in his original form to the video renderer for        display.

The two individual parts are two separates processes that areoperational on two distinct computers or CPU systems.

The key, if unique, can provide information to only one group of usersthat have access to the unique information.

The key can have an unlimited life duration time, or can be limited intime. For example, if the media is a pay per view program, it ispossible to generate a key that allow the end user, or the group of endusers to play the video only for a limited time. If the delay ofvisualisation expired, then the end user is not allowed any more to playthe content, and the player application can display a warning message.

The frame source data size has to be a multiple of 16 pixels for thewidth and the height. This is majority the case for all video alreadyencoded using compression algorithms that fit this restriction.

The ENCODED DATA and the SOURCE DATA have the same volume of pixels,i.e. the same volume of patterns. The aspect ratio parameter of theENCODED DATA can be the same than the SOURCE DATA, but it is notnecessary. To make the task much complex for the user that wish toaccess to the SOURCE DATA, we modify the aspect ratio of the ENCODED.DATA (VIDEO). The shape, the number of pixels and patterns are still thesame (rectangle), but the measurement in width and height are different.

The aspect ratio of a two-dimensional shape is the ratio of its longerdimension to its shorter dimension. It is also applied to twocharacteristic dimensions of a three- dimensional shape, especially forthe longest and shortest ‘axes’ or for symmetrical objects (e.g. rods)that are described by just two measures (e.g. length and diameter). Insuch cases, the aspect ratio may evaluate to a value less than one (e.g.consider very short and very long rods).

The preferred application of the invention is known as to protect imagecontent for Equirectangular images or video files and streams, and canbe applicable to all others images or video contents. TheEquirectangular format is well known in the computer graphic industry

It is most appreciated that the image data that will be processedthrough the Encoder has to be in an uncompressed format. It can work aswell with compressed files, but the final result will not be as good asneeded for different fields of applications, including the Entertainmentindustry

Video compression typically operates on square-shaped groups ofneighbouring pixels, often called a Macroblock.

The data can be one or multiple still images or one or multiple videofiles. The final processed file or files exported from the Encoder canthen be stored or streamed directly to the Decoder in two differentforms:

As an original video file using standard video compression. The AVIcontainer (Audio Video Interleave) format or the new MPEG-4 formatallows all kind of compression files, hi this case, the encryption keyis processed and sent (in the case it is not embedded directly in theDecoder) separately as a metadata,

As a metadata file including video and data. Metadata is used tofacilitate the understanding, use and management of data. The metadatarequired for effective data management varies with the type of data andcontext of use. In the context of an information system, where the datais the content of the computer files, metadata about an individual dataitem would typically include the name of the field and its length.

Each frame of the video has to be processed frame by frame at a lowlevel, and the images need to be in a progressive scan format (notinterlaced) in 24 bits, 32 bits or indexed colors.

The present invention is not intended to be restricted to any particularform or arrangement, or any specific embodiment, or any specific use,disclosed herein, since the same may be modified in various particularsor relations without departing from the spirit or scope of the claimedinvention herein shown and described of which the apparatus or methodshown is intended only for illustration and disclosure of an operativeembodiment and not to show all of the various forms or modifications inwhich this invention might be embodied or operated

We claim:
 1. A method for encoding visual information comprising thesteps of: dividing the visual information into a predetermined number ofpatterns over a matrix having an original aspect ratio; determining theposition of each pattern on the matrix; rearranging the patterns intonew positions within the pattern of the original aspect ratiorearranging the patterns rearranged into new positions within theoriginal aspect ratio into a new pattern having a predetermined aspectratio that is different from the original aspect ratio to generateencoded visual information, generating an encryption key for use indecoding the encoded visual information,
 2. A method as claimed in claim1, wherein the visual information is extracted frame by frame frommultimedia data comprising audio information by filtering andtransmitting the audio without encoding.
 3. A method as claimed in claim2, wherein each frame is progressive to prevent compression anomaly. 4.A computer program product for encoding and decoding visual informationcomprising one or more computer readable media configured to: divide thevisual information into a predetermined number of patterns over amatrix; determine the position of each pattern on the matrix; rearrangethe patterns on the matrix in a predetermined shape over the matrixusing a fuzzy algorithm; and generate an encryption key using theoriginal visual information to arrange the patterns information in orderto determine the position in the new shape; wherein the area of thepredetermined shape is equal to the area of the original visualinformation.